-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes for large scale hyperopt #2083
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
ShreyaR
commented
Jun 2, 2022
•
edited
Loading
edited
- Enable windowing for large datasets
- Add support for tune.report while using ray backend
- Add support for distributed hyperopt
ShreyaR
force-pushed
the
large-scale-hyperopt-v2
branch
from
June 2, 2022 08:22
26067d4
to
7a3db8c
Compare
ShreyaR
commented
Jun 2, 2022
ludwig/data/dataset/ray.py
Outdated
if not enable_windowing: | ||
pipe = self.ds.repeat() | ||
else: | ||
if window_size_bytes is None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An alternative to requiring window_size_bytes
to always be set is to calculate window size based on the following variables:
- Total object store memory in the ray cluster, and
- If running from within a tune trial and piping through the max concurrent trials parameter.
Not sure if that would be overkill tho so I've implemented a simpler solution for now.
tgaddair
reviewed
Jun 2, 2022
tgaddair
reviewed
Jun 2, 2022
ShreyaR
force-pushed
the
large-scale-hyperopt-v2
branch
from
June 2, 2022 21:39
df66d84
to
78c4ac0
Compare
ShreyaR
force-pushed
the
large-scale-hyperopt-v2
branch
from
June 2, 2022 22:34
23a11d9
to
459e039
Compare
ShreyaR
force-pushed
the
large-scale-hyperopt-v2
branch
from
June 6, 2022 10:40
a567256
to
3999703
Compare
tgaddair
reviewed
Jun 6, 2022
ShreyaR
force-pushed
the
large-scale-hyperopt-v2
branch
from
June 6, 2022 22:32
9a669cb
to
2589297
Compare
for more information, see https://pre-commit.ci
tgaddair
approved these changes
Jun 6, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.